1. Field of the Invention
The invention generally relates to management of SAS wide ports. More specifically, the invention relates to methods and structures for ensuring proper ordering in processing of received frames in a SAS device coupled through a SAS wide port.
2. Discussion of Related Art
Small Computer Systems Interface (“SCSI”) is a set of American National Standards Institute (“ANSI”) standard electronic interface specifications that allow, for example, computers to communicate with peripheral hardware. Common SCSI compatible peripheral devices may include: disk drives, tape drives, Compact Disc-Read Only Memory (“CD-ROM”) drives, CD Read/Write (“CD-RW”), digital versatile disk (“DVD”) drives, printers, scanners, etc. SCSI as originally created included both a command/response data structure specification and an interface and protocol standard for a parallel bus structure for attachment of devices. SCSI has evolved from exclusively parallel interfaces to include both parallel and serial interfaces. “SCSI” is now generally understood as referring either to the communication transport media (parallel bus structures and various serial transports) or to a plurality of primary commands common to most devices and command sets to meet the needs of specific device types as well as a variety of interface standards and protocols.
The collection of primary commands and other command sets may be used with SCSI parallel interfaces as well as with serial interfaces. Examples of serial interface transport media and protocol standards that support SCSI command processing include: Fibre Channel, Serial Bus Protocol (used with the Institute of Electrical and Electronics Engineers 1394 FireWire physical protocol; “IEEE 1394”) and the Serial Storage Protocol (SSP).
SCSI interface transports and commands are also used to interconnect networks of storage devices with processing devices. For example, serial SCSI transport media and protocols such as Serial Attached SCSI (“SAS”) and Serial Advanced Technology Attachment (“SATA”) may be used in such networks. These applications are often referred to as storage networks. Those skilled in the art are familiar with SAS and SATA standards as well as other SCSI related specifications and standards. Information about such interfaces, media, protocols and commands is generally obtainable at the website www.t10.org.
Such SCSI storage networks are often used in large storage systems having a plurality of disk drives to store data for organizations and/or businesses. The network architecture allows storage devices to be physically dispersed in an enterprise while continuing to directly support SCSI commands directly. This architecture allows for distribution of the storage components in an enterprise without the need for added overhead in converting storage requests from SCSI commands into other network commands and then back into lower level SCSI storage related commands.
A SAS network typically comprises one or more SAS initiators coupled to one or more SAS targets via one or more SAS expander devices. In general, as is common in all SCSI communications, SAS initiators initiate communications with SAS targets. In particular, SAS initiators use a process often referred to as “discovery” to determine the topology of devices in the network (i.e., to discover other SAS initiators, SAS expanders and SAS targets). Once such information is known, initiators generally establish the first contacts with a given target device. The initiator issues an “open” request (i.e., a SAS OPEN address frame) to an identified SAS target to establish a first connection with the SAS target device. Once the first connection is so established, either the SAS initiator or the SAS target device may re-establish a connection. For example, a connection may be established initially by the initiator, closed after some transactions are exchanged, and then re-opened by the same initiator for a subsequent sequence of transactions. Or, for example, a SAS target device may have deferred processing of a transaction received from an initiator. At some later time when the SAS target is ready to proceed, the target device may “open” a connection back to the initiator that originally requested the deferred transaction.
In SAS protocol command exchanges, a SAS initiator device sends a command and associated data to a particular identified SAS target device. The command and data exchanged may be voluminous but the SAS specifications limit each frame to approximately 1 kilobyte. Thus a larger exchange must be broken into smaller frames and distributed over a number of frames. Further, since the SAS specifications permit multiple commands/data to be exchanged concurrently. Thus each SAS command is identified with a unique TAG field value. This TAG field value is provided in the headers of each transmitted/received frame so that the receiver may determine which command a received frame is associated with.
SAS specifications also allow for a wide port—a logical port comprising a plurality of physical or link ports (i.e., multiple PHYs) aggregated to operate as one logical port. SAS specifications restrict use of the wide port such that at any given point in time, frames for a particular command may be transferred over only one of the multiple links that make up the wide port at a time. Once one frame is completely transmitted, another frame may be received on the same link or another of the multiple physical links of the wide port. When frames are received, they are typically temporarily stored in buffers associated with the physical link on which the frame was received. Eventually, when the transport layer is ready to process a next received frame, a frame will be transferred from its buffer to the transport layer for further processing. Since the processing of received frames may proceed slower than the speed at which the frames are received, there may be a sequence of received frames scattered in buffers of the various physical links of the SAS wide port.
The received frames in the buffers of the physical links are eventually forwarded up to higher level processing in the SAS controller—i.e., to a transport layer processing element. It is critical that the received frames be forwarded to the transport layer in proper sequential order. In some prior designs, the transport layer and physical or link layer (i.e., the PHY layer) may be tightly coupled in a single processing element. In such a tightly coupled design of the multiple layers of the SAS protocol processing, the sequencing of received frames may be easily maintained between the link layer and the transport layer—i.e., the two layers readily process the same SAS information in received frames. However, in many current designs, more of the processing of the link layer processing element is performed in custom designed circuits to attain higher performance. In such a design, the transport layer may be physically and logically separated from the link layer. The two layers communicate through an interconnecting communication medium and may, for example, be coupled through industry standard interface bus structures such as PCI or AHB/AMBA bus structures. In such designs where the transport and link layers are more separated than integrated, it is more difficult for the transport layer and the link layer to share information. Thus it can be problematic to assure proper sequencing of the processing of received frames as the link layer processing elements are more distinct and separate from the transport layer processing element.
In view of the above discussion, it is evident that there is an ongoing need for improved systems and methods for assuring proper sequencing in the processing of SAS frames received over multiple physical ports of a SAS wide port.